1. Field of the Invention
The present invention relates to a moving image coder for adaptively switching intra frame coding and inter frame coding.
2. Description of the Related Art
An adaptive inter frame image coding method is known when moving image information is digitally transmitted or recorded. In this coding method, intra frame coding and inter frame coding are adaptively switched and used. A differential signal between an input signal and a predicting signal is normally inputted to a coder as it is in a inter frame coding mode. However, in the case of a scene change, etc., an input signal is directly inputted to a coder as it is in a intra frame coding mode.
FIG. 8 is a block diagram of a general adaptive inter frame image coder.
A intra frame block X as a coding object block is inputted to an image input terminal 61. A difference between the intra frame block X and a predicting block P is calculated by a subtracter 62. The predicting block P is calculated by making a motion prediction from a reference image held in a predicting section 71. Thus, an inter frame differential block E is obtained by an output of the subtracter 62.
A control section 64 compares the intra frame block X and the inter frame differential block E with each other and judges whether the intra frame block X or the inter frame differential block E is coded. The control section 64 then controls the operation of a switch 63 in accordance with this judgment. In the following description, an intra frame coding mode is called when the intra frame block X is selected. An inter frame coding mode is called when the inter frame differential block E is selected.
A block D selected by the switch 63 is inputted to a coding section 65 and is coded by this coding section 65 so that code data Y are outputted from this coding section 65.
Coding mode information C relates to selection of one of the intra frame block X and the inter frame block E. The coding mode information C and a motion vector MV in the motion prediction are multiplexed to the code data Y by a multiplexer 66. A coding output Z of the multiplexer 66 at a multiplexing output terminal 67 thereof is transmitted or recorded.
A decoding operation is simultaneously performed on a coding side by using the code data Y and the coding mode information C.
The code data Y are inputted to a decoding section 68 and are decoded by this decoding section 68 so that a block D' is obtained. The block D' and the predicting block P are inputted to two input terminals of an adder 69 so that a sum of these blocks is calculated by this adder 69. A block E' as results of this additional calculation is obtained as an output of the adder 69.
Each of the blocks E' and D' is selected by a switch 70 controlled by the coding mode information C. At this time, the block D' is selected when the intra frame block X is inputted to the coding section 65 in the intra frame coding mode. The block E' is selected when the inter frame differential block E is inputted to the coding section 65 in the inter frame coding mode. The selected block is set to a decoding block X' and is inputted to the predicting section 71.
A complete decoding image is obtained when processings of one picture (one frame) are terminated. This decoding image is set to a reference image for generating a predicting block with respect to a subsequent coding object and is held in the predicting section 71.
In general, each of images located before and after a coding image in time is used as the reference image. When the motion prediction is used, a generating coding amount in the inter frame coding mode is smaller than that in the intra frame coding mode in a case in which an object is moved in parallel, etc. so that a portion corresponding to a coding image block exists within the reference image. Accordingly, it is preferable to select the inter frame coding mode in this case. In contrast to this, when an object appears from a screen end or a place behind something and scenes are changed, etc., no portion corresponding to the intra frame block exists within the reference image. In this case, the generating coding amount in the intra frame coding mode tends to be smaller than that in the inter frame coding mode. Accordingly, it is preferable to select the intra frame coding mode in this case.
The intra frame coding mode and the inter frame coding mode in the control section 64 are ideally switched in accordance with large and small coding amounts generated in codings of images if distortional degrees of these images caused by the codings are equal to each other. Namely, a smaller one of the generated coding amounts is selected in this case. In contrast to this, if the coding amounts generated by the codings are approximately equal to each other, the intra frame coding mode and the inter frame coding mode in the control section 64 are ideally switched in accordance with large and small distortions caused by the codings. Namely, a smaller one of the distortions generated by the codings is selected in this case. However, no coding amounts generated in the intra frame coding mode and the inter frame coding mode can be known in advance before the codings are performed. Further, no distortions caused by the codings can be accurately known before the codings are performed.
Therefore, for example, a mean square error of the intra frame block and the predicting block is compared with variance of the intra frame block in the literature of "A predicting system between MPEG2 frames" described in Television Society Technical Report, vol. 16, No. 61, pp. 37-42. This mean square error of the intra frame block and the predicting block and this variance of the intra frame block are respectively simply called the mean square error and the variance in the following description. If the variance &lt;64 is formed, the between-frame coding mode is used. In contrast to this, if the variance &gt;64 is formed, a mode providing min the mean square error, the variance! is selected. In this case, min x, y! means that a smaller one of x and y is used.
Here, variance of the inter frame differential block instead of the mean square error should be originally compared with the variance of the intra frame block. However, it can be supposed that an average value of pixel values in the inter frame differential block is equal to zero. Further, the variance of the inter frame differential block is obtained by subtracting the average value of pixel values in the inter frame differential block from the mean square error. Accordingly, the mean square error is used in the above literature. The average value of pixel values in the inter frame differential block can be supposed to be zero since it is known that a distribution of differential values between pixels having a high correlation with respect to an image is generally equal to a Laplace distribution having an average value of zero.
FIG. 9 is a block diagram of a calculating section for calculating the variance of the intra frame block.
Pixel value data in the intra frame block are stored to a cache memory 75 and are read from this cache memory 75 in accordance with an address generated by a controller 87. When a pixel value in the intra frame block is set to x(n), the variance of the intra frame block is represented by the following formula (1). ##EQU1##
In this formula, n shows an address of the cache memory 75 and N shows the number of pixels within the intra frame block. "-" shows an average calculated in an average calculation within the intra frame block.
In FIG. 9, a constructional portion 76 surrounded by a dotted line calculates the following value ##EQU2## as a mean square value of pixel values in the intra frame block.
An output of pixel value data of the cache memory 75 is inputted to two input terminals of a multiplier 77. Accordingly, a square value of the pixel value data is outputted from the multiplier 77 to one input terminal of an adder 78. An output of the adder 78 is inputted to an input terminal of a register 79. An output of the register 79 is inputted to another input terminal of the adder 78.
The adder 78 and the register 79 constitute an accumulative adding section. The output of the register 79 is also inputted to a latch 80. The latch 80 is used to hold calculated results and an operation of this latch 80 is controlled such that only results of the accumulative addition in a block unit are transmitted to a subsequent circuit portion. When these results are held, contents of the register 79 are reset on the basis of a control output of the controller 87 to calculate an accumulative adding value in the next block. When the number N of pixels is written as N=2.sup.a by power of 2, a calculation divided by this number N can be made by performing only a bit shifting operation in which a dividend is rightward shifted by a-bits. Accordingly, a divider is omitted in FIG. 9. In a case except for this case, it is necessary to arrange a divider after the latch 80. An output of the latch 80 shows a mean square value of pixel values in the intra frame block.
In FIG. 9, a constructional portion 82 surrounded by a dotted line calculates the following value ##EQU3## as a square value of the average value of pixel values in the intra frame block.
The output of pixel value data of the cache memory 75 is inputted to one input terminal of the adder 83. An output of the adder 83 is inputted to an input terminal of a register 84. An output of the register 84 is inputted to another input terminal of the adder 83. The adder 83 and the register 84 constitute an accumulative adding section. The output of the register 84 is also inputted to a latch 85. The latch 85 is used to hold calculated results and an operation of this latch 85 is controlled such that only results of the accumulative addition in a block unit are transmitted to a subsequent circuit portion. When these results are held, contents of the register 84 are reset on the basis of a control output of the controller 87 to calculate an accumulative adding value in the next block. As mentioned above, a calculation divided by the number N of pixels can be made by performing only a bit shifting operation in which a dividend is rightward shifted by a-bits. Accordingly, a divisor device is omitted in FIG. 9. In a case except for this case, it is necessary to arrange a divider after the latch 85.
An output of the latch 85 is inputted to two input terminals of a multiplier 86. Accordingly, the multiplier 86 outputs a square value of an average value of pixel values in the intra frame block.
Finally, the output of the latch 80 is inputted to a minuend input terminal of a subtracter 81. The output of the multiplier 86 is inputted to a subtrahend input terminal of the subtracter 81. The subtracter 81 outputs a value of the variance of the intra frame block.
In the following description, a pixel value of the intra frame block is set to x(n) and a pixel value of the predicting block is set to y(n) when n is set to an address of the cache memory. In this case, a mean square error of the inter frame differential block is represented by the following formula (2). ##EQU4##
This mean square error corresponds to a case in which x(n) is set to a pixel value of the inter frame differential block in the formula (1) for calculating the variance and the following equality condition is satisfied. ##EQU5##
Accordingly, x(n)-y(n) as a difference between the pixel value x(n) of the intra frame block and the pixel value y(n) of the predicting block is read from the cache memory. The read difference x(n)-y(n) is inputted to a mean square value calculating circuit composed of a multiplier, an adder, a register and a latch equal to those in the constructional portion 76 shown by a dotted line in FIG. 9 so that this mean square error can be calculated.
FIG. 10 is a block diagram of a coding mode judging section when the above mean square error is used. This coding mode judging section is arranged in the control section 64 in FIG. 8.
A mean square value calculating section 76 and a section 82 for calculating a square of an average value in FIG. 10 respectively correspond to the constructional portions 76 and 82 in FIG. 9. The mean square error and the variance respectively calculated with respect to the inter frame differential block E and the intra frame block X are inputted to a comparator 89. Coding mode information C is determined and outputted by large and small relations of the mean square error and the variance.
As can be seen from FIG. 9, a product sum calculation and a sum product calculation are required to calculate the variance. It is necessary to arrange at least two multipliers and three adders-subtracters. In particular, each of the multipliers is a factor causing a great increase in hardware scale of the moving image coder. The product sum calculation is also required to calculate the mean square error. In this case, it is necessary to arrange at least one multiplier and one adder. Accordingly, three product sum arithmetic units and three sum product arithmetic units are required to judge the inter frame coding mode and the intra frame coding mode so that a circuit scale of the moving image coder is very large.